Simulation studies of the reliability of computer aided model specification using the TETRAD, EQS and LISREL programs

TETRAD n, a fully automated successor to the TETRAD program, is intended to aid in the respecification of underspecified linear causal models, or structural equation models. The performance of TETRAD II is compared with the automatic respecification procedures in the EQS and LISREL VI programs using 360 simulated data sets from nine different linear models containing "latent" or unmeasured variables. For these cases, we find that the TETRAD II program, which uses graph algorithms and heuristic search techniques, is significantly more reliable than either EQS or LISREL VI, which use numerical algorithms and beam search techniques. A detailed analysis of the reasons for these differences is offered. Contrary to those who dismiss automated search techniques as unreliable "ransacking" or "data mining," TETRAD II provides correct information about the true model for 93.8% of the large sample data sets. The need for further simulation tests and the prospects for the development of automated techniques to aid in the initial specification of causal models for nonexperimental data are discussed. An appendix illustrates the application of the published TETRAD program to one of the data sets in this study. The research reported in this paper was supported by the Office of Naval Research under Contract number N00014-88-K-0194.